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A New Evaluation Approach for Teacher Preparation Programs 
Using Labor Market Competitiveness of Teacher Applicants 


Peter Goff, Hyunwoo Yang, Minseok Yang, Lena Batt, Xin Xie, and Eunji You 


Abstract 


This study introduces a new approach to measure effectiveness of teacher preparation programs 
at U.S. universities (TPP) by examining to what extent TPPs produce employable teacher 
candidates. We use teacher application data in Wisconsin public schools from 2014-15 through 
2016-17. We find that attending a specific university’s TPP makes a difference for novice 
teacher applicants’ job outcomes. However, the competitiveness of a TPP is inextricable from 
the geographic locale in which it operates. These findings call into question the validity of 
acontextual and absolute TPP rankings. Our results suggest that graduates of TPPs with strong 
school and community partnerships are more competitive in the local labor market. 
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Introduction 


Teacher preparation programs (TPPs) are the institutions where prospective teachers obtain 
foundational knowledge regarding subject matter and pedagogy, and begin to hone these skills 
through classroom experiences before starting their teaching professions (Feuer et al., 2013). 
These learning and teaching experiences are essential contributors to the quality of instruction, 
particularly for novice teachers. Along with substantial empirical evidence validating the 
intuition that the quality of instruction is important for student learning outcomes (Hanushek & 
Rivkin, 2010; Kane et al., 2013), interest has been increasing interest in whether and to what 
extent TPPs prepare the future teaching force to work effectively in public K-12 schools. Many 
educators and policymakers believe that it is necessary to evaluate the effectiveness and quality 
of these programs for the improvement of teacher preparation, teacher quality, and ultimately 
student performance (Feuer et al., 2013). 


State education agencies and various policy groups have developed TPP evaluation systems. 
The federal government has sought to ensure the quality of TPPs by requiring them to report 
data, such as graduation rates, average of certificate test scores, and grade-point average (GPA), 
and thereby producing a report card for each TPP, per Title II of the U.S. Higher Education Act. 
State governments responsible for program approval decisions also have their own review 
systems or partnerships with national accrediting bodies, such as the Council for the 
Accreditation of Educator Preparation. Although the evaluation systems vary across states and 
policy groups (e.g., National Council on Teacher Quality, Education Next), the evaluation 
criteria reflect combinations of similar evidence, including program selectivity (e.g., ACT scores 
or GPA), faculty qualifications, student teaching experience (e.g., minimum hours of student 
teaching), surveys of program graduates and employers, teacher certification scores and pass 
rates, hiring and retention rate of the graduates, and aggregate of graduates’ scores of value- 
added models representing the impact on student learning. 


Although the measures may differ, all evaluation systems focus on the extent to which TPPs 
produce effective teachers who can improve student learning. However, the prevailing evaluation 
strategies can miss significant aspects of the school context and the TPP’s integration with the 
teacher labor market. First of all, the teacher education system and teacher labor market are 
segmented by grades and disciplines. It is possible that a TPP may have strengths in art and 
social studies education, but may be comparatively weaker in math education. In such a case, the 
TPP would be marked as a low-performing institution in many of the current evaluation systems, 
especially through VAMs that assessed only through student math and reading scores. Similarly, 
in the teacher labor market, teacher candidates compete with others in the same subjects and 
adjacent licenses. The current evaluation systems have been conducted with aggregated criteria 
that label TPPs as effective, mediocre, or failed, without considering the differences among the 
different subject areas. Importantly, the majority of TPP evaluation approaches compare TPPs 
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only through the performance of TPP graduates who enter the classroom and thus neglect to 
account for the selection effects during hiring that separate the stronger candidates from the 
weaker. 


This study introduces a new approach, measuring to what extent TPPs produce employable 
teacher candidates. During the hiring process, employers strive to select the best candidate by 
taking applicants’ comprehensive qualifications into account, including potential ability to 
increase students’ test scores and attitudes, prior experience, collegiality, and specific school 
needs. Recruiting, selecting, hiring, and on-boarding teachers is an expensive process, as are the 
processes needed to remove an ineffective teacher. These costs encourage schools to be aware of 
which candidates have and have not been successful within their organization; over time this 
awareness informs hiring process. With the assumption that high stakes and high costs of teacher 
hiring coupled with historical-organizational knowledge make employers’ hiring decisions an 
effective tool with which to evaluate teacher candidates, TPP effectiveness can be closely related 
to TPP competitiveness in the localized and segmented job market. In other words, among many 
candidates, those who are ultimately employed as teachers can be regarded as among the most 
effective teachers, through the eyes of the schools in which they were hired, relative to the other 
candidates in the applicant pool. With the same assumption, this paper argues that TPPs that 
produce graduates who are more likely to be employed, relative to graduates of other programs 
vying for the same position, are more effective. 


To demonstrate the validity of the novel approach, this study uses statewide applicant data 
for three academic years and adopts vacancy fixed effects model with additional statistical 
controls. As a primary analytic model, the vacancy fixed effects model enables us to measure the 
probability of being hired among the candidates who applied to the same vacancy. In addition, 
through the inclusion of a distance variable between TPPs and the school posting the vacancy, 
we illustrate manner in which TPPs are embedded in their local teacher labor markets. This 
finding suggests that the competitiveness, and thus effectiveness, of TPPs is geographically 
relative, and calls into question the extent to which existing evaluation systems’ absolute 
rankings of TPPs are valid. Finally, this study investigates the characteristics of TPPs that are 
related to the competitiveness for policy discussions. The following research questions are 
addressed: (1) Is a uniform ranking of teacher preparation programs valid? (2) How does a 
localized teacher labor market impact the competitiveness of a teacher preparation program? (3) 
How does specialization impact the competitiveness of a teacher preparation program? 


Literature Review 


Evaluations of Teacher Preparation Program 

In the era of accountability, as teachers and schools are expected to increase student 
performance, TPPs also have been expected to produce high-quality teaching force (Zeichner, 
2010). For instance, Title II (Section 205) of the Higher Education Act (HEA) includes 
provisions requiring all states and TPPs that receive federal assistance under the HEA to report 
information on the performance of their programs (Kuenzi, 2018). This mandate has led 
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researchers and policymakers to create strategies to evaluate the effectiveness of TPPs. The 
underlying premise of such evaluation efforts is that high-quality TPPs will produce high-quality 
teachers (Ballou & Podgursky, 2000; Cochran-Smith & Zeichner, 2005; Crowe, 2010; Levine, 
2006; Vergari and Hess, 2002). The fundamental purpose of such evaluation efforts is to create 
formative feedback across TPPs by identifying effective preparation practices and to suggest 
ideal models of TPPs in order to improve the overall TPP quality (Feuer et al., 2013). This 
expectation has created a variety of evaluation methods. A neo-liberal interpretation of this 
policy would see that the purpose lies in the information contained in the evaluation ratings that 
will inform future teachers’ enrollment decisions and will allow the most effective institutions to 
flourish and the weaker ones to close. Another thought is that such evaluation systems will be 
used in a summative manner by government agencies to direct resources toward or away from 
TPPs, on the basis of their TPP evaluation ratings. 


Much like approaches to teacher evaluation (Peterson, 2000), there are two broad categories 
of TPP evaluation: input and output measures. Input measures range from easily attainable 
variables, like admission and recruitment criteria, to more subjective measures including the 
quality and substance of instruction, faculty expertise, and quality of student experiences. 
Although these input measures can be easily obtained and compared between TPPs, scholars 
contend that input measures are not reliable to gauge the effectiveness of TPPs (Fuller, 2014). 
For example, Boyd, Lankford, Loeb, Rockoff, and Wyckoff (2008) cautioned that materials in 
syllabi and course listings are not always directly related to the material and instruction provided 
in programs. Coggshall, Bivona, and Reschly (2012) argued that surveys on student teaching 
experiences might not facilitate comparison of experiences among different programs, due to the 
subjective nature of surveys and self-reporting. Similarly, faculty qualification has not been 
proven to be correlated with outcomes of teacher candidates (Feuer et al., 2013). 


Output measures used to evaluate TPPs range from comparisons of teacher licensure tests 
(e.g., Praxis tests) or pass rates to value-added modeling (VAM) of recent graduates. However, 
licensure test scores may not be feasible when comparing between and within states due to the 
different test cut-off levels (Fowler, 2001; Crowe, Allen, and Coble, 2013; Sawchuk, 2013). 
Although some evidence suggests that candidates’ performance assessments in the form of 
observations or video-taping can predict their future effectiveness in the classroom (Darling- 
Hammond, Newton, & Wei, 2013), Greenberg and Walsh (2012) explain that this method has 
not been established as a valid or reliable measure for determining the quality of a TPP because 
it is likely that teachers manipulate the outcome measures by choosing, rehearsing, and editing 
their performance. 


The most common and increasingly popular outcome measure is to evaluate the effectiveness 
through VAM (Goldhaber, 2019). Recent policies encourage VAM to measure TPP effectiveness 
through aggregating performances of program graduates (Harris, 2011). For example, and Race 
to the Top, which features a grant competition to award states aimed at developing and 
implementing innovative evaluation system for TPPs, requires student outcomes as measured by 
VAM to be included as the measurement of TPP effectiveness (Feuer et al., 2013). 
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However, VAM is not a flawless indicator for TPP effectiveness (Bastian, Patterson, & Pan, 
2018). Existing studies demonstrated substantial differences in the value-added scores between 
TPPs (e.g. Boyd, Grossman, Lankford, Loeb, & Wyckoff, 2009; Goldhaber, Liddle, & Theobald, 
2013; Henry et al., 2014), while others showed negligible differences (e.g., Harris & Sass, 2011; 
Von Hippel, Bellows, Osborne, Lincove, & Mills, 2016; von Hippel & Bellows, 2018). Multiple 
studies found that variations in TPPs do not lead to more effective teachers; in fact, greater 
variation exists within programs compared with between programs (Goldhaber, 2019). von 
Hippel et al. (2016) found no significant differences between TPPs in Texas when widening 
confidence intervals to include teacher random effects. von Hippel and Bellows (2018) expanded 
this analysis to six states and found similar results with only a small number of individual TPPs 
being statistically differentiable from the others. Scholars also argued that VAM produces biased 
estimation (Gansle, Noell, & Burns, 2012; Goldhaber et al., 2013; Rothstein, 2009). For instance, 
it is difficult to disentangle TPPs effects from school context factors. In addition, students’ test 
scores may not be sensitive enough or be the right medium to identify differences between 
program components and programs in general, especially when not all subjects and grade levels 
are tested (Goldhaber, 2019). VAM measures also fail to capture the vast majority of recent 
graduates, the majority of whom do not teach in subjects or classes that can be assessed via 
VAM. 


A recent policy interest in TPP evaluation has been advanced to consider a broader set of 
outcomes, including job placement rates, retention, employer/graduate surveys, and graduate 
teacher evaluation ratings like VAM (Bastian et al., 2018). For instance, state-level regulations 
(lasevoli, 2017) and the Council for the Accreditation of Educator Preparation (CAEP; 2013) 
have provided more comprehensive measures of TPP outcomes to facilitate program 
accountability and ongoing improvement. However, these multi-outcome systems are in an 
nacent state and the implications of this new way forward is unclear (Bastian et al., 2018). In 
sum, existing TPP evaluation approaches present notable flaws which have led policymakers, 
educators, and prospective teachers to be reluctant to take the evaluation results to improve 
quality of TPPs as originally intended. Ultimately, there is ample room and needs to develop 
additional strategies to improve upon comprehensive TPP evaluation systems. 


Decision Making in Teacher Hiring 


Schools that are successful in attracting and hiring effective teachers have greater growth in 
student performance (Grissom & Loeb, 2011; Loeb, Kalogrides, & Beteille, 2012). Thus, school 
personnel who are involved in recruitment strive to identify the highest quality teachers to fill the 
vacancies in their schools. However, there are many ways for teachers to be viewed as high- 
quality teachers in the eyes of potential employers. Indeed, principals who are one of the most 
important actors within a school organization that regularly interacts with teachers, do not 
necessarily choose the most academically competitive and credentialed candidates when they fill 
a vacancy (Baker & Cooper, 2005). Instead, principals assess teacher quality by holistically 
synthesizing all available information about an applicant and contrasting holistic assessment 
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against the needs and preferences of their school (Harris, Rutledge, Ingle, & Thompson, 2010; 
Rutledge, Harris, & Ingle, 2010). 


Prior studies have found that school personnel are more likely to hire teachers showing 
enthusiasm (Dunton, 2001), interpersonal skills (Jacob & Lefgren, 2005), caring personality 
(Rutledge et al., 2010), strong communication skills (Cain-Caston, 1999; Dunton, 2001), strong 
teaching skills, and subject matter knowledge (Rutledge et al., 2010). Also, prior experiences, 
including student teaching and TPP affiliation are typically seen as key factors for new teachers 
by recruitment and hiring teams (Boyd, Lankford, Loeb, Ronfeldt, & Wyckoff, 2011; Rutledge, 
Harris, Thompson, & Ingle, 2008). Furthermore, studies demonstrate that district and school 
contexts such as district and school size (Levin & Quinn, 2003; Liu, 2007), labor union policies 
like collective bargaining agreements (Strunk & Grissom, 2010), accountability pressures 
(Harris, 2012), student racial/ethnic composition, and socio-economic status in neighborhood 
and school contexts (Broadley & Broadley, 2003) can influence how employers conceptualize 
teacher quality. In addition, employers may prefer job candidates that fit well into the existing 
contexts of their schools including cultural norms, organizational values, and other school 
members while bearing specific skills needed for the vacancy (Harris et al., 2010; Liu & 
Johnson, 2006; Papa & Baxter, 2008). In sum, the perceived effectiveness and quality of teacher 
candidates can vary widely across vacancies. 


In this regard, many TPP evaluations, particularly the outcome measure models that purport 
to assess the overall ‘quality’ of their graduates, fail to capture TPP effectiveness. Rather than 
relying on an acontextual measure of teacher quality (e.g., VAM), it may be fruitful to consider 
tapping into the knowledge and expertise latent within the system. As Jacob and Lefgren (2005) 
showed, principals are reliable agents to evaluate teacher effectiveness, in part because they 
identify effective teachers in a holistic matter by integrating multiple sources of related 
information. Through their professional practices, school leaders may take note of skills, 
knowledge, and supports that accompany teachers from particular TPPs. Presumably, if some 
TPPs consistently provide a competitive advantage for their graduates in the labor market, the 
advantages could be interpreted as a signal of TPP effectiveness that may comprehensively 
reflect not only the input and output measures but also the school contexts and fit. Furthermore, 
the signal of TPP competitiveness would be more evident for novice candidates than experienced 
ones because novice teachers have fewer sources of information for screening, such as prior 
teaching experience and VAM scores. If the competitive advantages in the labor market could be 
appropriately estimated, they may be incorporated into TPP evaluation systems. Should there be 
meaningful, differential signal obtained from this approach, ratings could be used to provide 
TPPs and prospective students important information to shape policy, programs, and practice 
(Feuer et al., 2013). 


Organizational Learning 

Organizational learning theory can help convey how the competitive advantages in a labor 
market could inform TPP evaluation. Organizational learning theory posits that institutions 
acquire knowledge through information acquisition, sharing and dissemination, shared 
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interpretation, and the development of organizational memory (Huber, 1991; Slater & Narver, 
1995; Tippins & Sohi, 2003), similar to the ways of individual learning processes. As learning 
organizations, schools also develop their learning processes, learning strategies, and knowledge 
structure, which enables them to effectively adapt to changes in their external environmental 
(Giles & Hargreaves, 2006). Schools engage in this organizational learning in a cumulative 
manner, using ongoing experiences, such as teacher hiring and retention, to continuously update 
their knowledge base. Those organizational experiences are stored and coded into organizational 
memory (Walsh & Ungson, 1991; Schechter & Mowafagq, 2013), and organizational members 
draw the encoded past information to guide present decision-making processes (Kruse, 2003). 


Importantly, organizational memory can comprise soft information, such as tacit knowledge, 
experiences, anecdotes, and stories that are embedded in members’ memories, organizational 
culture and norms, as well as hard data such as numbers, facts, and figures that are stored in a 
form of written documents, electronic database, historic archives and so on (Kruse, 2003; 
Morrison, 1993). From this perspective, schools can retain knowledge on which new hires were 
most effective based on localized past experience, with some knowledge being stored by 
principals and other forms of knowledge residing in teacher-leadership teams or in the revised 
protocols of the district’s human resources department. Accordingly, if a newly hired teacher’s 
TPP affiliation is systematically associated with their success (or failure), the high-stakes of 
these choices encourages the organizational memory of schools to retain this information as well. 
This organizationally acquired knowledge would be applied and updated each hiring cycle, 
providing some TPPs with consistent competitive advantages in the labor market. 


In addition, prior research of organizational learning has found that the rate of learning varies 
depending on a number of factors. The characteristics of the organization, such as the member’s 
proficiency, organizational structure, technology, and routines as well as method of coordination 
are key determinants of how well organizational learning occurs (Argote, 2012). Also, 
environmental contexts influence the different rates of learning such as regulators, marketing 
environments (including competitors), and clients (Argote, Miron-Spektor, 2011), and an 
environment of psychological safety (Edmondson, 1999). As with the idea of different rates of 
organizational learning, the learning may occur more intensely and efficiently with knowledge 
pertaining to high-stakes decisions. In the teacher labor market literature, high stakes of hiring 
decisions mean that districts have a strong incentive to make the best hiring decision due to high 
fiscal and temporal costs resulting from unsuccessful teacher recruitment (National Commission 
on Teaching and America’s Future, 2003). Therefore, tapping the latent knowledge within a 
large number of organizations regarding the relative quality of TPPs by observing their responses 
to high-stakes hiring decisions may be a viable strategy to evaluate the effectiveness of TPPs. 


Although several researchers have conducted empirical studies regarding organizational 
learning in the educational realm by investigating the relationships between degree of 
organizational learning and teachers’ information usage (Schechter, 2007), school environment 
promoting organizational learning (Klein, 2000), and principals’ sense of uncertainty and the 
extensiveness of organizational learning in schools (Schechter & Asher, 2012), no study thus far 
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has explored how organizational learning is linked to teacher hiring. The process of 
organizational learning, organizational memory, and retrieval, may play an important role in 
shaping perceptions of teacher education programs for school personnel, which results in 
differential preferences across TPPs. 


Prior literature suggests that hiring agencies do not seriously consider TPP classification 
(e.g., highly competitive) and that experiences in TPPs (e.g., student teaching, course taking) are 
not critical in recruiting processes (Baker & Cooper, 2005; Ballou, 1996; Engel, 2013; Engel & 
Curran, 2016). While prospective employers may not be attuned to broad classifications of TPPs, 
they may well care about individual TPPs through their organizational memory that is 
established through professional and organizational networks as well as their past experience. 
For instance, student teaching, which is the most common way to shape relationships between 
TPPs and districts, can facilitate the organizational memory in schools about TPPs if past and 
current student teachers from the same TPP consistently impress school personnel with similar 
performances over time. TPPs coordinate student teaching, and most placements are near the 
TPPs (Greenberg et al., 2011). In that teacher hiring occurs with poor information about 
candidates (Cannata et al., 2017), school employers actively utilize their organizational memory 
because hiring previous student teachers in their schools is one of the safest ways to avoid an 
unsuccessful match and to select candidates who are likely familiar with school contexts (Engel 
& Finch, 2015; Rutledge et al., 2008). Goldhaber, Krieg, and Theobald (2014) found that 
districts hire nearly 15% of student teachers as soon as they complete their internships. 


In addition, given that teacher labor market is narrowly localized (Engel & Cannata, 2015; 
Harris et al., 2010; Reininger, 2012), organizational learning could manifest through the 
relationships among local resources such as faculty, administrators, and alumni in close 
proximity to the TPPs where they graduated. Some research found that the most successful 
principals tend to have substantial social capital within the school district and successfully utilize 
their network sources when hiring new teachers (Cannata et al., 2017). Therefore, the social 
capital of principals connected with the TPPs may play a vital role in establishing a recruitment 
pipeline. For instance, in Chicago Public Schools, the district statement of teacher hiring policy 
explicitly encourages principals to utilize opportunities for student teaching and university 
relationships to build up a recruitment network (Engel & Curran, 2016). In this sense, TPP 
evaluations through a lens of organizational learning theory is closely related to the main concept 
of jurisdictional advantage that indicates a place-specific advantage in competitions (Feldman & 
Martin, 2005). In other words, competitive advantages of TPP in the job market tend to be place- 
specific, advantages determined through organizational learning within a local boundary, rather 
than stemming from TPP classifications using selectivity and evaluation of value-added models. 
The value an organization may place on a TPP graduate can change based on proximity, 
exposure, and interaction. 
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Method 


Data and Sample 


Our primary source of data are teacher candidates’ job application data provided by the 
Wisconsin Education Career Access Network (WECAN) for the 2014-15 through 2016-17 
school years. WECAN is the online job portal system for both administrators and teachers in 
Wisconsin K-12 public schools. The data include academic backgrounds and credential 
information for applicants as well as vacancy descriptions for school districts. To the best of our 
knowledge, WECAN is the only statewide data source in the U.S. that capture detailed labor 
market activities from both supply and demand sides. 


We limited the sample to novice teacher candidates who graduated from TPPs within the 
three school years. Using the licensure information provided by the Wisconsin Department of 
Public Instruction, we identified the applicants who received their first license within the three 
years preceding their job search. We also restricted the sample to vacancies that hired novice 
teachers. We chose these approaches because when novice and experienced teachers go into the 
job market, the presence of the experienced teachers in the application pool could distort the 
impact of TPPs on the novices’ hiring probability. To illustrate, an experienced teacher may have 
a higher probability of being hired through accumulated social networks with schools/districts, 
skills and knowledge accrued through their years of experience, or seniority-based placement 
policies. Focusing on novice teacher candidates and vacancies that hired these applicants 
excludes many confounders coming from experienced teachers and allows us to isolate the 
contribution of TPP affiliation on hiring probability. Collectively, the data capture 119,742 job 
applications submitted by 8,317 candidates who graduated from Wisconsin TPPs to 6,731 
vacancies posted by 376 districts. 


Our outcome variable is whether the candidates were hired to the vacancy where they 
applied. Because WECAN does not provide any information regarding hiring results, we 
combined the vacancy/application data with Wisconsin Department of Instruction staff records, 
which describe where teachers work and the subject they teach, allowing us to infer which 
candidate was hired. 


The predictor of interest is the TPP where an applicant completed their training and obtained 
their teacher license. There are 14 public, 21 private, and eight alternative-route TPPs in 
Wisconsin (Wisconsin Department of Public Instruction, 2017). We did not include teachers who 
completed their preparation through alternative routes in the sample because WECAN does not 
provide that information. We also excluded applications from candidates who completed their 
education from TPPs with fewer than 30 graduates per year, because the low N of these 
institutions prevents reliable estimation. Seven TPPs were dropped from our analytical sample 
after this adjustment. The final sample included 12 public and 14 private TPPs, which accounted 
for 93.4% of all novice applicants. Those who graduated from TPPs outside Wisconsin (e.g., 
Minnesota and Illinois) are categorized into one group (out-of-state), which is the reference 
group for the empirical investigation. 
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We also created a series of control variables that capture the demographics of applicants in 
our sample, namely, (a) an indicator whether an applicant is female; (b) an indicator whether an 
applicant is a person of color; (c) years of professional experience, and (d) GPA. We used GPA 
as a proxy for teacher quality, which is more appropriate for beginning teachers and accounts for 
any variations in grade inflation across TPPs. 


Of the 8,317 applicants, 75.6% are female and 4.6% are teachers of color, as shown in Table 
1. The applicants report an average of 3.2 years of related professional experience prior to 
obtaining licensure and entering the job market. These experiences include student teaching, 
tutoring, and time in other education-related positions. The applicants’ average GPA is 3.62. The 
summary Statistics reveal that their demographics are not notably different from the current 
teachers’ demographics in Wisconsin public schools. 


Table 1 shows that applicants, on average, submitted 14 applications to vacancies in seven 
school districts. Median values are 6 and 4, respectively, suggesting some applicants submit 
many applications whenever relevant vacancies appear. Table 1 also illustrates that each vacancy 
receives an average from 18 applications of novice teachers from eight TPPs. Given the analytic 
sample has 26 TPPs dispersed across the state, Table 1 indicates that each vacancy on average 
has graduates from one third of TPPs as a part of their application pool. 


Table 1. Descriptive statistics for teachers and vacancies 


Mean SD Min Max 
Applicants (N=8,317) 
Female 0.76 (-) 0 1 
Teachers of color 0.05 (-) 0 1 
Professional experience oek7 5.03 0 10 
GPA 3.62 0.33 0 4 
Number of applications submitted 14.4 23.5 1 370 
Number of districts where applied 7.03 9.29 1 114 
Vacancies (N=6,731) 
Number of applicants 17.79 21.48 1 253 
Number of TPPs 7.57 5.2L 1 26 


Notes: Professional experience indicates all teaching-related experience (e.g., student 
teaching, boy/girl scout activities) before an applicant receives teaching license. 


Empirical Framework 
To examine the impact of TPPs on teacher hiring, we estimate linear probability models of 
the following form: 


Hired;, = Bo + Br TPP. + Xiv9 + by + Ej, (1) 


where Hired equals one if applicant i was hired in vacancy v; TPP is a dummy indicator of 26 
TPPs (reference group: out-of-state); X is a vector of individual teacher characteristics; 5 
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represents vacancy fixed effects; and ¢ is a random disturbance term. Bx are the primary 
coefficients of interest in Equation 1, representing the estimated labor market advantage 
associated with each TPP, as compared to the reference group (out-of-state). 


The TPP dummy variable TPP, represents all TPP-related institutional characteristics (e.g., 
student enrollment, curriculum, location). The variable also captures different teacher labor 
market demands around TPPs such as the number of available vacancies. The vacancy fixed 
effects (5) allow us to explore the hiring probability of applicants within a vacancy in a given 
year, rather than simply comparing all applicants across vacancies. This approach mitigates the 
bias resulting from unobservable district-level vacancy-level characteristics (e.g., different labor 
market demand and various personnel policies, different labor market contexts). 


In some of our analyses, we are interested in whether TPPs have relative competitiveness in 
specific geographic areas. We divided the sample into two groups: vacancies at (a) elementary 
schools and (b) nonelementary schools, followed by the first analysis. This approach reveals 
heterogeneous competitiveness within a TPP based on the labor market results. One possible 
concern is that an aggregate measure of TPPs does not take into account the different 
competitiveness across subjects within a TPP and may not capture the true TPP competitiveness. 
For example, one TPP may be renowned for its music education program, while the other 
programs are not as effective. Although splitting the sample by subject fields (e.g., math, science, 
special education, music, and art) may be the ideal approach to handle this inquiry, it is not 
possible due to the small sample sizes and consequent statistical power issues. Thus, we divided 
the sample into two groups of elementary (the largest sub-group) and nonelementary. 


To explore the extent to which hiring probability varies by the distance (i.e., miles) between 
the TPP where an applicant graduated and the school districts to which the applicant applied, we 
included the distance variable in the model as follows: 


Hirediy = Bo + Bx, TPP, + 6, TPP; * Distancey, + Xj,8+ 6, + Ein (2) 


where, TPP*Distance is an interaction term between the TPP indicator and the distance 
illustrated above. All other terms are as defined in Equation 1. Based on the perspective from 
organization learning and localized teacher labor market, we hypothesized that the interaction 
coefficients would have negative values, suggesting the hiring probability decreases as the 
distance from the applicant’s TPP increases. To support an intuitive understanding of TPP 
competitiveness, we calculate marginal values based on the above model and present the 
predicted hiring probabilities of each TPPs in the figures below. 


Results 


Teacher Preparation Programs and Teacher Hiring 


Figure 1 shows how a novice teacher applicant’s TPP affiliation relates to that applicant’s 
probability of being hired. Each dot represents the average predicted hiring probability 
associated with the corresponding TPP, with vertical bars indicating 95 percent confidence 
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intervals. Each row of Figure 1 corresponds to one of the 27 TPPs in our final sample. The 
results show that hiring probabilities of applicants significantly differ by from which TPP they 
graduated. For example, graduates from TPP 1 have the highest predicted probability of being 
hired at 13.7%, while TPP 27 has the lowest probability of 4.9%. In other words, an applicant 
who graduated from TPP 1 is more likely to be hired than an applicant from TPP 27, by 8.8 
percentage points, if those two applicants compete for the same position. Although not every 
TPP has statistically distinct probability (e.g., TPPs 7 through 12 show similar hiring 
probabilities with overlapping confidence intervals of their predictive margins), the overall 
picture suggests that where a teacher candidate received their training may make a substantial 
impact on their likelihood of being hired. 


Figure 1. Predicted Hiring Probability by Teacher Preparation Programs 


0 .03 .06 .09 12 Bt les 
Hiring Probability 
To explore the heterogeneity in TPP competitiveness, we divide the sample into two cases: 

(a) elementary school vacancies and the applicants who applied to such positions (shown in the 
left panel of Figure 2) and (b) nonelementary school vacancies and the applicants who applied to 
such positions (shown in the right panel of Figure 2). Figure 2 shows that the aggregate TPP 
competitiveness presented in Figure 1 does not change dramatically when the sample is 
disaggregated by the type of vacancy. We see here that TPPs that provide a labor market 
advantage to their graduates with elementary licensure also provide a comparable labor market 
advantage to their graduates in other certification areas. TPP 1 still represents the highest hiring 
probability in both elementary and nonelementary vacancies while TPP 27 also indicates the 
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lowest hiring probability in both groups. We find that average hiring probabilities across TPPs 
are lower in case of the elementary vacancies than the nonelementary vacancies, reflecting the 
larger number of applicants per elementary vacancy. The average number of applicants per 
elementary vacancy is 28 (= 81,023/2,866) while the number decreases to 10 (= 38,719/3,865) 
for nonelementary vacancies. Novice candidates face more competition when they vie for 
elementary positions as compared to nonelementary vacancies. This difference in the applicant 
pool corresponds to the findings from a recent report on teacher supply and demand in 
Wisconsin (Goff et al., 2018). We also attribute the large standard errors in the nonelementary 
vacancies to the difference in the number of applications (81,023 vs. 38,719) as well as to the 
heterogeneity of position types represented (e.g., art, history, automotive). 


Figure 2. Teacher Preparation Programs and Hiring Probabilities 
in Elementary and Nonelementary positions 
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Localized Teacher Labor Market 


Our second research question addresses whether geographic proximity is associated with a 
competitive advantage for applicants vying for positions close to their TPPs. To illustrate the 
interaction between distance and TPP fixed effects we constructed predictive margins using five 
distance measures specific to each TPP (as shown in Figure 3). These distance measures 
represent various percentiles (10th, 25th, 50th, 75th, and 90th) from each TPP to the districts to 
which the TPP’s graduates applied. We adopt this approach, rather than using the average 
percentiles across all TPPs, because the application distance differs substantially across TPPs, as 
shown in Table 2. For example, for some TPPs in urban areas, most applications from their 
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graduates targeted nearby urban and suburban vacancies, which are within fairly small radius of 
the TPP. On the other hand, some TPPs are in town or rural areas, and their graduates show 
substantially broader search radii. Table 2 presents the variation in search radii by TPP. For 
example, we note that the median distance for applicants of TPP 4 is 13 miles, whereas those 
from of TPP 25 is notable larger (246 miles). Given the different application radii corresponding 
to the locations of TPPs and the labor market demands of districts, we calculated the marginal 
effects using distances that were appropriate for each TPP. 


Table 2. Average Distance between TPPs and Districts, by TPPs 
Distance between TPPs and Districts by Percentile (Ptile) Ranges 


TPP Ptile 10th Ptile 25th Ptile 50th Ptile 75th Ptile 90th 
1 2.8 2.8 9.3 38.4 91.4 
2 1 10.3 97.6 172.5 206.7 
3 2.8 9.3 13.4 49.7 82.2 
4 3 73 13 23 61 
fs) 1.3 18.4 43.1 73.3 91.7 
6* = a = 7 - 

Vi 6.2 10.1 43 W2A 99.3 
8 29.5 33.6 48 84.3 111.2 
9 0.4 9.1 14.7 49 84.2 

10 hie 11.8 26.9 66.7 80.7 
11 4.8 9.2 23.4 67.3 86.8 
12 ae 10.8 37.8 102 134.2 
13 4.5 6.4 13.6 27.9 59.4 
14 29.8 60.3 90 112.2 130.6 
15 18.2 26.7 3739 48.6 97.9 
16 9.9 70.2 112 160.1 171.8 
17 5.3 26.7 51.3 105.5 128.7 
18 6.8 10.3 18.6 47.9 83.2 

19 18.2 23 40.3 58.8 86.2 

20 21.5 pails) 93.8 192:5 227.5 
21 49 58 82.4 130.9 151.4 
22 18.4 59.5 132.2 225 254.5 
23 14.2 22.9 45.8 74 93.7 
24 14.1 24.6 36.9 49.1 103 

25 85.6 134.7 246.4 281.9 324.8 
26 10.9 70.7 111.7 161.3 174.2 
27 10.6 35.9 53.8 81.7 100.5 
All 6.7 18.4 44.7 90.3 149.5 


Note. TPP 6 represents out-of-state programs, which do not have the distance measures and 
are not included in this distance interaction model. 
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The interaction between TPP fixed effects and distance measures, represented by the 
predictive margins in Figure 3, shows that most of the TPPs present declines in their graduates’ 
hiring probabilities as their graduates apply to vacancies farther away from the TPPs. In Figure 
3, “x” indicates the hiring probability at the median distance range (same as 50% in Table 2). For 
example, in the case of a graduate of TPP 16, the hiring probability is 9% at the median distance, 
which is 11 miles. The range of hiring probability is represented by the white and light gray 
markers being to the right of the “x” (closer and more likely to be hired) and the black and dark 
gray markers falling to the left (farther and less likely to be hired). TPP 1 shows the highest 
hiring probability (16%) when their graduates apply to the nearby vacancies, whereas the 
estimates decrease (7%) for graduates applying to vacancies farther away. Their hiring 
probabilities increase up to 13% when the graduates apply to vacancies whose distance is 10 
percentiles. 


Figure 3. Heterogeneous TPP Effects Depending on Distance 
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From a close examination of Figure 3 we see that, although the average ranking of TPPs 
appears similar to the ranking of Figure 1, proximity to TPP emerges as a game-changing factor 
in the labor dynamic. For example, applicants from TPP 26 tend to have a lower than average 
probability of being hired (roughly 7%). However, when these applicants apply close to their TPP 
(within 11 miles or so), their applicants are hired at rates comparable to the most competitive 
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TPPs (roughly 12.5%). This proximity effect roughly doubles applicants’ chances of being hired. 
Similarly, TPP 12, at rank 3, holds a notable labor market advantage for their candidates with an 
average hiring probability of 11.5%. Yet when these graduates apply to vacancies farther away 
(190 miles), their probability of being hired drops precipitously to among the lowest in sample 
(4%). Collectively, this pattern suggests that effectiveness of a TPP is not an absolute that can be 
easily extracted, measured, and compared. Rather, TPP effectiveness is a quantity that is 
inherently relative to and inextricable from the context in which a program operates. 


Interestingly, the labor market advantage of three TPPs 3, 21, and 27 appear not to change 
notably with distance, and two TPPs 13 and 22 show an inverse trend, in that the hiring 
probabilities for their graduates increase as their graduates apply to vacancies farther away. The 
former suggests that some TPPs may have a reputation and networks that extend uniformly 
across the state, while the latter suggests that the reputation and networks that a TPP develops 
locally may not always be advantageous to graduates. 


Last, we find that distance asymmetrically affects the relative competitiveness of TPPs. Some 
TPPs (e.g., TPP 1, 12, 22) show huge variations in the hiring probabilities depending on distance 
while some TPPs (e.g., TPP 4, 10, 25) are notably smaller. These TPPs perhaps have closer 
relationships with the districts closer to them, and the connection may lead to positive labor 
market outcomes for their graduates. The finding also suggests that some TPPs (e.g., TPP 3) 
maintain a similar reputation across the state while others (e.g., TPP 27) fail to maintain 
reputation in nearby and distant districts. Because our analytic model employs the TPP dummy 
variables to examine the hiring competitiveness, we cannot capture why these differences arise. 
In the discussion section, we return to our theoretical framework and investigate various 
characteristics of TPPs and suggest possible interpretations. 


Discussion 


Scholars previously identified criteria to measure or evaluate TPPs while media and 
organizations ranked programs, essentially inferring the possibility that preparation programs, 
regardless of context, can be absolutely compared. Most of these evaluative criteria included 
inputs (e.g. admission criteria or faculty expertise; Fuller, 2014) or outputs (value-added models; 
Harris, 2011) but researchers have critiqued these evaluation methods and identified how the 
criteria may not capture TPP effectiveness. In this study, we examine TPPs in a different way by 
employing the concept of organizational learning in school-level hiring for novice teachers who 
have just graduated from their preparation program, essentially making their education a major 
component of their marketability. We posit that the hiring process is informed by organizational 
learning, which means that schools rely on previous experiences, sharing and dissemination of 
information, and organizational memory to select new teachers and continue partnerships 
(Huber, 1991; Slater & Narver, 1995; Tippins & Sohi, 2003). A school’s prior experiences with 
graduates from a particular TPP play a role in the hiring of new teachers and these continued 
partnerships are likely part of the organizational learning in which schools engage. As a result, 
we used application and staffing data for novice teachers in Wisconsin to understand whether a 
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specific TPP’s alumni are more competitive than those graduating from another program and 
how program heterogeneity and the local teacher labor market play roles. 


Our findings demonstrate that attendance at a particular TPP can make a difference when 
novice teachers apply for jobs, but the local market and program characteristics must be 
considered as factors in this process. When graduates apply for jobs close to their TPP, their 
hiring probability typically increases, relative to their chances when applying to a school farther 
from their preparation program. When comparing an overall ranking of TPPs across Wisconsin 
we find that schools toward the bottom of the list are almost as competitive as the top schools 
when looking at their marginal probability of being hired by a school in close proximity to their 
TPP. The findings call into question whether scholars, policymakers, or administrators should 
place value in an overall ranking of TPPs. 


We first investigated whether particular programs within TPPs affected the competitiveness 
of graduates by comparing TPP alumni hiring for elementary and nonelementary vacancies. We 
found no significant difference in TPPs between vacancy level, but the available amount of data 
limited our ability to investigate differences among programs such as special education or 
specific disciplines (e.g. secondary mathematics), which may have produced more significant 
findings. The similarity in TPP graduate competitiveness between elementary applicants and all 
other applicants may indicate that the central practices used and values imparted by a TPP are 
universal across all their programs. When a common approach for teaching literacy, for example, 
is used across all programs and is deployed by a common group of faculty, such a possibility 
becomes more plausible. In larger institutions programs may be more balkanized and isolated 
from each other, making such commonalities less likely. This question should be pursued when 
more data are available to investigate programmatic differences. 


Next, we examined the impact of distance on TPP graduate competitiveness in the teacher 
labor market. School leaders use practical experiences such as student teaching to hire novice 
teachers in an effort to find a candidate who is more familiar with and fits into the school context 
(Engel & Finch, 2015; Rutledge et al., 2008), which can impact the local teacher labor market. The 
selection process exemplifies how schools can use their organizational learning and memory to use 
the limited knowledge they have in a hiring process that often occurs with poor information 
(Cannata et al., 2017). We argue that since TPPs coordinate most student teaching opportunities, 
and most placements are near the TPPs (Greenberg et al., 2011), these partnerships influence the 
competitiveness of the alumni of a particular program and the local teacher labor market. The 
manner in which TPPs collaborate with schools, districts, and other local education agencies, is 
understudied. Our findings here can provide direction so such inquiries will yield fruitful findings 
that can inform how TPPs provide value to their graduates and the surrounding communities. 


We reviewed websites for the TPPs at the top and at the bottom of the list according to 
predictive margins of hiring when distance from TPP to district is included (Figure 3) to better 
understand our findings. The primary similarity across the top five TPPs (1, 5, 12, 2, and 8) was 
that each touted their extensive network of community and school partners and the hundred plus 
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hours that preservice teachers spent in educational settings before beginning student teaching. 
We suggest that K-12 schools’ organizational learning and memory is shaped by these extensive 
partnerships, making surrounding districts more likely hire new teachers from these institutions. 
The TPPs toward the bottom of the list (22, 25, 27) also illustrated this inference. Of the TPPs 
whose alumni we found to be least competitive, our online search revealed that each had a 
prominent online component. We conjecture that if students are enrolling in an online TPP, they 
do not likely live near the university and therefore may not be able to take advantage of local 
partnerships. With a large number of online students, the TPP likely does not place as high a 
value on developing school partners and/or may lack the opportunities to cultivate substantive 
relationships. Second, the schools at the bottom of our list were often located close to a state 
border, and the websites stated that the programs prepared students to work in the neighboring 
state and highlighted their proximity to the larger cities outside of Wisconsin. Our data only 
capture Wisconsin school vacancies and the graduates of these schools may be searching for jobs 
in other states. We conclude the TPPs with strong school and community partnerships are more 
competitive in the local market because school leaders are more willing to hire the candidates 
they know. Additional qualitative research is needed to further investigate the role of 
partnerships and organizational learning in TPP competitiveness. 


Counter to our initial hypothesis, we identified one case in which program characteristics could 
impact a TPP’s competitiveness beyond the local labor market. For example, preparation programs 
13 and 22 (Figure 3) had an inverse relationship for distance compared to other schools, in which 
graduates from those TPPs were more competitive farther from the school rather than within their 
local market. We reviewed the websites of both TPPs and found one programmatic difference 
common to both universities: Both TPPs trained students in the Montessori method of education. 
The majority of Wisconsin’s Montessori schools were located across the state from the two TPPs 
and might be a reason for the competitive edge of graduates as they applied farther from the TPP. 
While overall heterogeneity in programs did not significantly impact competitiveness of TPP 
alumni, programs such as Montessori preparation could make a difference. As with all the 
aforementioned TPP evaluation strategies, our approach is not without flaws. However, the 
corroboration of the inferences we have made from our findings with secondary data collected on 
the characteristics of the TPPs lends credence to the validity of our approach. 


In conclusion, our findings suggest a new way of evaluating TPPs that refutes noncontextual 
and absolute rankings. At a time when the efficacy of TPPs are called into question with the rise of 
alternative pathways to teaching (Darling-Hammond et al., 2005), policymakers should be 
thoughtful about how TPPs are evaluated and measured. The current evaluation systems may 
mislead policy efforts, such as Title II of the Higher Education Act, to improve the overall quality 
of TPPs by inaccurately identifying good and bad programs. Continuing to rank TPPs, either 
through media such as U.S. News and World Report or through institutions such as the National 
Council of Teacher Quality, gives a false sense that it is possible to absolutely rank TPPs in terms 
of their alumni’s competitiveness on the job market. Our findings argue that evaluators of TPPs 
should consider their role in the community and their network of partners they cultivate. These 
relationships can be a key to their graduates being locally competitive in the job market. 
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